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ABSTRACT 


Refereeing is a crucial component of publishing astronomical research, but few professional 
astronomers receive formal training on how to effectively referee a manuscript. In this article, we lay 
out considerations and best practices for referees. This document is intended as a tool for early career 
researchers to develop a fair, effective, and efficient approach to refereeing. 


1. INTRODUCTION 


Referees are responsible for assessing the quality and 
novelty of research and to provide feedback to the edi- 
tor on whether the result is appropriate for the journal. 
They are also responsible to provide feedback on the 
framing and presentation of the results. An effective 
referee will respond with a fair, kind, and actionable 
report to the authors and to the journal editor. 

Because they provide support and feedback for assess- 
ing new scientific results, referees are a vital part of sci- 
entific publication. And though this is an important 
part of our scientific careers, few astronomers receive 
formal training in the process. 

In this document, we discuss the process of referee- 
ing, including: understanding ethical conflicts, framing 
the referee report, avoiding common referee pitfalls, nav- 
igating rejections, and framing the final referee report. 
See Wager et al. (2002); Nicholas & Gordon (2011); Raff 
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(2013), and references therein, for additional informa- 
tion and complementary perspectives on refereeing best 
practices. 

This reference is intended primarily for referees-in- 
training as they learn how to referee journal articles. 
It may also be useful to mentors as a tool for coaching 
their early-career mentees through the process of becom- 
ing an effective referee. 


2. UNDERSTANDING ETHICAL CONFLICTS 


Before agreeing to referee a manuscript, consider eth- 
ical conflicts that might prevent a referee from being a 
fair referee or that might damage your professional rep- 
utation and relationships. 


1. Am I a close collaborator of any of the authors? 
Engaging in an anonymous, critical conversation 
with close collaborators may damage these profes- 
sional relationships. 


2. Do I have a poor relationship with any of the au- 
thors? Even if you feel that you can give an un- 
biased assessment of their work, remember that 
there are other valid referees who would not be 
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spending a large amount of emotional energy to 
fairly assess the work. 


3. Is it “too close to comfort”? If the research is 
so close to your own work in progress that you 
could not give honest feedback or suggestions for 
improvement, you should discuss this with the ed- 
itor before proceeding. 


4. Do I have the expertise to properly assess this re- 
search? This can be particularly difficult in the 
case of work that draws from more than one dis- 
cipline, and any concerns about interdisciplinary 
work should be directed to the editor before you 
agree to review the manuscript. 


If you suspect that you cannot be a fair, objective, and 
fully invested referee for any reason, you should write 
a short note to the editor to discuss your concerns; the 
editor can help you to evaluate this. 


3. QUESTIONS FOR CONSIDERATION 


Expectations and norms vary from journal to jour- 
nal (e.g. American Astronomical Society 2022; MNRAS 
2022; Nature 2022), and when possible, you should con- 
sult with the editorial staff about what qualities should 
be considered in your review. Assessment questions we 
have found useful to focus on are: 


1. Is the research sufficiently innovative? Does it 
bring something new to the field? 


2. Do the authors put the research in sufficient con- 
text? Is their literature review sufficient? Have 
they missed any citations? 


3. Are the data and methods clearly described? Is 
there enough information to recreate the authors’ 
analysis? 


4. Have the authors provided sufficient information 
to make their results reproducible, including ac- 
cess to data and software? The referee should as- 
sess whether the authors have an adequate plan 
for sharing data and software with the community 
and should share this assessment with the editor. 


5. Are the methods used appropriately? 
6. Are there passages that are unclear or ambiguous? 


7. Have the authors provided sufficient evidence to 
support their claims? Have the authors made un- 
substantiated claims? Have they overstated their 
results? 


8. Have the authors appropriately discussed caveats 
and limitations? Have they provided clear expla- 
nations for any results that seem too good to be 
true? 


While it is not necessary to answer each of these ques- 
tions explicitly, these are just some of the assessments 
that will frame your report. 


4. AVOIDING COMMON PITFALLS 


As a referee, you should not make unkind criticisms 
about topics outside of the research and manuscript you 
are assessing. Do not make statements about the au- 
thors or their qualifications. 

You should not serve as a copy editor. If significant 
language issues make it difficult to assess the scientific 
content of the manuscript, pause your review. Recom- 
mending “proofreading by a native speaker” is inappro- 
priate. Instead, you should immediately contact the ed- 
itor, tell them your concerns, and let them reach out to 
the authors. If the main body of the manuscript is suffi- 
ciently understandable but there are typos or ambiguous 
passages, commenting on these is fine. 

As a referee, you should not expand the scope of the 
manuscript or push the manuscript to deviate signifi- 
cantly from the authors’ intentions. You are not the au- 
thors’ academic advisor and it is inappropriate to change 
the direction or significantly expand the scope of the 
manuscript. There are two exceptions to this: 1. if the 
original scope is insufficient for publication, you can and 
should explain this and consider recommending ways to 
extend the work, and 2. If you see a golden opportunity 
for the authors to improve their work, you might offer 
this suggestion but be clear that this is not a require- 
ment for publication. 

You should not write meandering prose regarding your 
opinions on the topic. The authors will need to respond 
to each item in your report, and this report is most 
useful when it is focused on clear, actionable items. 


5. NAVIGATING REJECTIONS 


It can be more difficult to recommend rejection than 
to recommend revision and resubmission. In the case of 
rejections, you will need to write a clear and kind note to 
the authors explaining the shortfalls of the manuscript, 
including strong evidence supporting your claim that it 
is not appropriate for the journal. This does not need 
to be exhaustive, but you do need to be specific about 
some critical flaws. It can be difficult to write this in a 
way that is both kind and constructive, but this should 
be your goal. Feedback that is not constructive can be 
included in the confidential response to the editor. 
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6. FRAMING THE FINAL REPORT 


Begin your review by explicitly describing the 
strengths, aims, and results of the manuscript. This 
sets a good tone for the constructive criticism that fol- 
lows and it communicates to the authors that you un- 
derstand their research goals. By including a summary 
of the manuscript and listing its strengths (rather than 
focusing only on the manuscript’s shortcomings), you 
communicate fairness. Using “the manuscript” instead 
of “the author” or “you” in your feedback is one way to 
keep your feedback neutral. 

Organize your feedback. The following template is 
commonly used: 


1. State the aims and key results of manuscript. 
2. Summarize the strengths of the manuscript. 


3. Summarize your constructive feedback or assess- 
ment of the manuscript. 


4. State your recommendation for publication. 


5. List the major weaknesses (e.g., methodological 
issues or overstated results). 


6. List the minor weaknesses (e.g., missing references 
or figure formatting issues). 


Clearly articulate your feedback. Refer to sections and 
line numbers where applicable, and provide sufficient 
publication information (or arxiv or ADS links) for the 
authors to quickly find references. 


7. FINAL COMMENTS 


Before you submit your report, you should edit the 
first draft of your report for tone. This is particularly 
important because emails often come across as more 
harsh than they are intended. Aim for polite discussion 
with constructive criticism and clear, actionable items. 
Your report should be courteous without compromising 
honesty. 

Referees provide an important service to our scientific 
community: evaluating new research and providing feed- 
back to ensure that the research is clearly presented and 
appropriately framed. As a referee, your goal should be 
to provide the type of report that you would want to 
receive. Strive to write a report that is a fair, kind, and 
actionable assessment of the research. 
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